Chronos User Guide
Updated: Oct 29th, 2024
Sections & FAQs
Where do I store my files on Chronos?
How can I transfer files to and from Chronos?
How can I check the current status of Chronos?
How do I use Python on Chronos?
How can I use Jupyter Notebook/Lab on Chronos?
Can I run my own Virtual Machine on Chronos?
Chronos: An Introduction
Chronos is our central lab server, primarily intended to support projects: (i) using Deep Learning/AI (ii) frequently handling large data-sets or which produce output files of significant size. Chronos currently has the following specifications:
OS: Ubuntu 22.04.3 LTS Kernel: 6.5.0-14-generic CPU: AMD Ryzen Threadripper PRO 5975 (32 Cores/64 Threads) GPU: NVIDIA RTX A6000 48GB RAM: 256GB* Backup Power: Eaton 9E2000I UPS**
Nevertheless, despite the main purpose of Chronos, the system is optimized for multi-user workloads and we encourage you to freely use the system when needed, or as your daily work-machine if you so wish. If Chronos is currently experiencing an extremely high workload (see: Section 7 - Status), please consult with those currently logged in to know when resources will be freed.
Most of all, Chronos was built with freedom in mind - the freedom to prototype and try new things without fear of a monthly bill.
*Chronos employs sophisticated memory compression techniques to significantly extend the capacity of the machine beyond 256GB of RAM.
**An uninterruptible power supply (UPS) device automatically provides backup power to Chronos during our frequent, but short-lived power cuts in the faculty. ~20-100 minutes of power is provided depending on workload.
Accessing Chronos
For security reasons, you must be present in the faculty, on campus or connected to the Technion VPN (https://cswp.cs.technion.ac.il/how-to-connect-using-vpn-to-technion-servers-from-outside-the-technion/) to access and use Chronos.
Chronos is an OpenSSH server, meaning you can access it via any SSH-client using the following details:
Host: 192.114.18.118
Port: 22Or directly on a SSH-enabled terminal using:
ssh username@192.114.18.118 #For example: ssh tim@192.114.18.118For Windows users, we recommend MobaXTerm as an all-in-one SSH client solution that will allow you to connect to Chronos but also manage/transfer files to and from the server (https://mobaxterm.mobatek.net/) or WinSCP for file management only (https://winscp.net/eng/index.php).
For R and Python users, equivalent access is also provided directly through RStudio (see: Section 8 - RStudio) and Jupyter Lab (see: Section 10 - Jupyter) respectively - without the need to connect separately.
In all cases, you will need to provide your username and password.
Changing Passwords
One of the first tasks we recommend you do after connecting to Chronos for the first time is to change your password. From any SSH-terminal (or via Rstudio/Jupyter Lab, see: Section 8 - RStudio and Section 10 - Jupyter) use the following command:
passwdAnd follow the instructions provided. If you forget your password, an administrator can reset it for you at anytime.
User Storage Space
Each user on Chronos has a private storage area that nobody else can access or modify, located at:
/media/chronos/Storage/[username] #For example: /media/chronos/Storage/timA shortcut (“UserSpace”) to this area is present in your /home/ directory (the default directory of your user upon logging in).
Transferring Files
Files can be transferred from an external computer to your UserSpace (and vice versa) in a multitude of ways. We recommend:
- Using an SSH-client e.g. MobaXTerm or SSH-enabled file manager e.g. WinSCP - which will allow you to drag and drop files to and from your UserSpace.
- Via RStudio itself (see: Section 8 - RStudio) which supports file upload/download.
Note: Optimal transfer speeds will be achieved when connected to the Technion LAN (i.e. in the faculty/on campus) and will be considerably slower when connected to the VPN at home.
Backups
Each UserSpace is incrementally backed up every 4h (starting daily at 00:00) to our Dropbox account.
Note: Backups are mirrors i.e. if you delete a file on Chronos, it is deleted on Dropbox. However, Dropbox will retain a restorable copy of all deleted files for 6 months. Moreover, Dropbox will store versioning information for each file, allowing files to be reverted back to an older copy (as far back as 6 months) when needed.
Files stored outside of your UserSpace are NOT backed up. It is therefore mandatory Chronos policy that all work, files, scripts and data are stored in this area.
In cases where a folder should not be backed up e.g. it contains installed packages, Python environments, a significant number of files (especially small files, e.g. >10,000) etc., we recommend excluding these from backup using the DNBU (Do Not Backup) function. For example:
DNBU add /media/chronos/Storage/tim/FolderToExcludeFromBackup #Exclude this folder and all contents from backupTo remove an exclusion or check the current exclusion status of a given folder:
DNBU remove /media/chronos/Storage/tim/FolderToExcludeFromBackup
DNBU check /media/chronos/Storage/tim/FolderToExcludeFromBackupServer Status
Once connected, Chronos will automatically report its current status to you. For example:
Server Status: Uptime: 0d|22h|47m MEMORY Memory (RAM): 112GB/251GB [==============> ] 44% # = Compressed Memory Disk Swap Memory: 0GB/50GB [> ] 0% DISK SPACE User Disk Space: 2643GB/9240GB [==========> ] 31% System Disk Space: 212GB/627GB [==========> ] 33% SSD Health: 100% CPU | GPU Average CPU Usage (5m): 0% [> ] 0% GPU (VRAM): 218MB/49140MB [> ] 0% CPU Temperature: 37.0°C | GPU Temperature: 35°C POWER Main Power: Online Currently Logged In: yochay merav tim
We recommend users to inspect this to gauge the current workload on Chronos, and time their own work accordingly. If, for example, many resources are coloured in red - the server is under heavy load and you may need to consult with the users currently logged in (shown above) to know when you can use Chronos.
At anytime, you can view updated status using the following command on any Terminal:
statusR Users - Using RStudio
Chronos runs RStudio Server 24/7 and the service does not need to be initiated by the user. To access RStudio from your own machine, simply direct your web browser to:
192.114.18.118:8787And login with your Chronos credentials.
This is a user-specific R environment. All packages installed are for you and you only - as are all RStudio settings and processes being run. You are responsible for updating packages and are free to uninstall/install packages at will.
RStudio Server provides a ‘Terminal’ tab which allows you to interact directly with Chronos as you would from any SSH-terminal. This allows you to, for example, change your password or check the current status of Chronos (see: Section 3 - Passwords and Section 7 - Status) .
Screenshot:
Using status and passwd commands via RStudio
To transfer files to and from Chronos using RStudio, use the ‘Files’ tab which provides an ‘Upload’ button and an ‘Export’ (download) button (in ‘More’).
Screenshot:
Upload/Download (export) using RStudio
Important: To free resources for active users, idle R sessions - sessions that have not executed code for >=3 hours - are written to disk and removed from RAM. When restoring your session (which RStudio will do automatically), please allow 1-5 minutes for it to transfer data from the disk back into RAM ready for use - the interface will not load until this has completed.
If RStudio fails to reload after a prolonged period of time - which can happen when a large amount of data was left inside your R environment - you can clear your saved session by connecting to Chronos outside of RStudio and issuing the command:
clearRTo avoid this and any loss of work, we recommend including code to save RObjects (e.g. .RDS files) throughout your scripts whenever running a large job and terminating your scripts with code that clears the REnvironment automatically:
rm(list = ls()) #Clear R environmentNote: no R session currently running code will be considered idle. In other words, you’re free to close your browser/laptop and Chronos will continue to execute your script to completion - no matter how long it takes.
To end a session, click the red ‘off’ button on the top-right of RStudio. This will also remove your current environment from both RAM and the disk.
The default working directory has been set to your UserSpace. If this is not the case, or the working directory was accidentally changed, please reset it:
setwd("/media/chronos/Storage/[username]") #For example: setwd("/media/chronos/Storage/tim")Unlike on Google Cloud, where each user must manage their own environment and install system packages necessary for R - this is handled centrally on Chronos.
All system packages typically required by R are already provided. Should you encounter additional needs, please contact an administrator.
Moreover, common R packages (including their dependencies e.g. ggplot2) have been pre-installed for each user:
CRAN/Github Packages:
devtools
remotes
Seurat
BiocManager
stringr
msigdbr
glmnet
stabs
ggpubr
BayesPrism
Bioconductor Packages:
SingleR
celldex
ClusterProfiler
tradeSeq
glmGamPoi
CATALYST
diffcytNote: R packages requiring access to Docker/Singularity e.g. dynverse are not supported on Chronos for security reasons.
Note: R environments e.g. the installed packages, are not backed up and cannot be restored in the event of removal. Please re-install packages instead.
Python Users - Using Conda
Chronos supports Conda for the creation and management of self-contained Python environments, and is accessible from any Terminal connected to Chronos. All Conda installations are user-specific, and cannot be accessed, run or modified by others.
We assume Python users are already familiar with the use of Conda, however a starter guide is available here: https://conda.io/projects/conda/en/latest/user-guide/getting-started.html#managing-python
While a more detailed guide is available here: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
If you’re new to Python, please contact an administrator who can teach you all of the necessary details. Not all users currently have conda enabled.
We recommend installing conda environments into your UserSpace. For example:
conda create --prefix /media/chronos/Storage/tim/Conda_Envs/EnvName python=3.11
conda activate /media/chronos/Storage/tim/Conda_Envs/EnvNameAs conda environments often contain ~200,000+ files, please exclude the corresponding folders from backup (see: Section 6 - Backups)
Instead, we encourage users to export a configuration file for each environment to another location in their UserSpace (which is backed up). For example:
conda activate /media/chronos/Storage/tim/Conda_Envs/EnvName
conda list --explicit > /media/chronos/Storage/tim/EnvName.ymlWhich allows for easy and rapid recreation of the environment:
conda create --prefix /media/chronos/Storage/tim/Conda_Envs/NewEnv --file /media/chronos/Storage/tim/EnvName.ymlLastly, you can update your own installation of conda using:
conda update condaPython Users - Running Jupyter
We recommend installing Juptyer Lab/Notebook into each, individual conda environment. For example:
conda activate /media/chronos/Storage/tim/Conda_Envs/EnvName
conda install jupyterWe typically recommend using Jupyter Lab, as opposed to Jupyter Notebook, as it provides additional functionalities including a ‘Terminal’ tab which allows you to interact directly with Chronos as you would from any SSH-terminal.
To access Jupyter, the service must be mapped to a user-specific port. To check which ports are available and select one to use, see here:
Add this port to your installation by first creating a Jupyter configuration file:
jupyter lab --generate-config #Replace 'lab' with 'notebook' if using jupyter notebook
nano ~/.jupyter/jupyter_lab_config.pyAnd adding the lines:
c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 5000 #Replace with your chosen portSave the file and launch Jupyter:
jupyter labThis configuration file will apply to Jupyter in all conda environments, however, if you wish to run multiple Jupyter instances simultaneously you will need to generate separate configuration files (each with unique ports) for each instance.
Virtual Machine Users - Using VirtualBox
For advanced users or specific needs, Chronos offers configurable VirtualBox-based Virtual Machines (VMs).
It is important to note, however, that VMs come with several, significant drawbacks:
- Virtual memory is non-swappable and non-compressible, and therefore VMs are limited to running within the confines of the hardware (256GB of RAM maximum). In other words, VMs cannot benefit from Chronos’ memory compression scheme. In practice, the maximum RAM a VM could use will be lower than 256GB given the multi-user nature of Chronos.
- VMs, unlike Conda, have no access to the GPU i.e. no hardware accelerated deep-learning.
- VMs employ lazy-allocation of RAM, but lack the ability to pass unused RAM back to the central system once it has finished using it. VMs therefore accumulate and retain resources overtime that could otherwise be freed for use by others.
For these reasons, we discourage the use of VMs as the vast majority of tasks can be accomplished without one while also taking advantage of all that Chronos has to offer.
Nevertheless, please speak to an administrator if you wish to setup and use a VM.
To manage your VM, once installed by an administrator, use the command:
vmhelperWhich provides an interface for common tasks, including backup of your VM(s) (which are not backed up automatically):
-------------------------------------------------
Currently installed VMs:
"Ubuntu" {d0bee6ae-a95c-46b8-8505-206aa1b94aae}
No VMs are currently running.
-------------------------------------------------
No VMs backups detected. Consider regularly backing up your VMs.
Boot VM [1], Shutdown VM [2], Modify VM [3], Backup VM [4], Cancel [5]
Please select an option:
All VMs run their own SSH server for direct access once booted.